Add native gem precompilation support#4080
Conversation
Package precompiled native gems for Ruby 3.3, 3.4, and 4.0 across 10 platforms: aarch64-linux-gnu, aarch64-linux-musl, aarch64-mingw-ucrt, arm-linux-gnu, arm-linux-musl, arm64-darwin, x64-mingw-ucrt, x86_64-darwin, x86_64-linux-gnu, and x86_64-linux-musl. - Add Ruby-version-aware extension loader in lib/prism.rb with GLIBC error message for musl/glibc mismatches - Add cross-compilation support to Rakefile using rake-compiler-dock - Add CI workflow for building and testing native gems across all platforms and Ruby versions - Add bin/ scripts for building, installing, testing, and verifying gems Also, temporarily switch to manual publishing via bin/build-gems + gem push, and remove automated publish-gem.yml workflow. We should plan on adding automated publishing of native gems once this all works.
The "gem" task does not exist.
|
Only minor comment (that can be left until after merge) is that I'd love these It seems like right now the GitHub action triggers on a bunch of stuff, but presumably after you get CI passing we'll switch it to just tags right? |
I don't think it's obvious that they should be rake tasks, but I do want to try to follow the project conventions. If that's what you want, we can try to make it work. The shell scripts are intentionally operating outside of bundler, and test-gem-file-contents is a self-contained minitest suite.
I mean, it's obviously up to you when you want to run these tests, but my implementation decision to run these tests on every pull request is intentional. The new workflow isn't publishing, it's testing:
and I think these are valuable tests for any PR that modifies the source code or the build process. |
flavorjones
force-pushed
the
native-gem-precompilation
branch
from
3108b0e to
aa1bf2d
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Nokogiri 1.19.1 and 1.19.2 include a call to recipe.activate in ext/nokogiri/extconf.rb that causes libiconv's installed .la file to record its libdir with Windows-style backslashes on aarch64-mingw-ucrt. When libxml2 subsequently links against libiconv, libtool parses that .la via eval and the backslashes are stripped, producing a garbled path and 'cannot determine absolute directory name' at CCLD libxml2.la. sparklemotion/nokogiri#3530 removed the recipe.activate call on main but has not yet landed in a release. Pin to main on the affected platform only, to unblock CI until a fixed nokogiri release ships.
flavorjones
force-pushed
the
native-gem-precompilation
branch
from
053f9e4 to
6d2af66
Compare
2 participants
WIP: Looking for CI feedback on this approach.
Closes #4026
Package precompiled native gems for Ruby 3.3, 3.4, and 4.0 across 10 platforms: aarch64-linux-gnu, aarch64-linux-musl, aarch64-mingw-ucrt, arm-linux-gnu, arm-linux-musl, arm64-darwin, x64-mingw-ucrt, x86_64-darwin, x86_64-linux-gnu, and x86_64-linux-musl.
Also, temporarily switch to manual publishing via bin/build-gems + gem push, and remove automated publish-gem.yml workflow. We should plan on adding automated publishing of native gems once this all works.
This approach uses rake-compiler-dock to build a broad set of platform gems, including GNU and Musl linux variants.
I also have an experimental branch that uses https://github.com/shopify/cibuildgem, however I think for a gem like prism it may be prudent to choose the more mature rake-compiler-dock toolchain, until cibuildgem is at parity.